Papers with random sampling
On Training Instance Selection for Few-Shot Neural Text Generation (2021.acl-short)
Copied to clipboard
| Challenge: | Pretraining large neural networks with a language modeling objective has led to dramatic improvements in text generation. |
| Approach: | They propose a selection strategy to select few-shot training instances based on unlabeled data to identify the most worthwhile data points that should be annotated under some budget of labeling cost. |
| Outcome: | The proposed strategy outperforms random sampling on three text generation tasks. |
Active Learning for New Domains in Natural Language Understanding (N19-2)
Copied to clipboard
| Challenge: | Existing approaches to improve the accuracy of new domains are lacking annotated live utterances. |
| Approach: | They propose an algorithm called Majority-CRF that uses an ensemble of classification models to guide the selection of relevant utterances and a sequence labeling model to prioritize informative examples. |
| Outcome: | The proposed algorithm achieves 6.6%-9% error rate reduction and statistically significant improvements on six new domains. |
Your Pretrained Model Tells the Difficulty Itself: A Self-Adaptive Curriculum Learning Paradigm for Natural Language Understanding (2025.acl-srw)
Copied to clipboard
| Challenge: | Existing curriculum learning approaches rely on manually defined difficulty metrics which may not accurately reflect the model’s own perspective. |
| Approach: | They propose a self-adaptive curriculum learning paradigm that prioritizes fine-tuning examples based on difficulty scores predicted by pre-trained language models (PLMs) they evaluate four datasets covering binary and multi-class classification tasks. |
| Outcome: | The proposed model leads to faster convergence and improved performance compared to standard random sampling. |
Data-driven Model Generalizability in Crosslinguistic Low-resource Morphological Segmentation (2022.tacl-1)
Copied to clipboard
| Challenge: | morphological segmentation is a common method of evaluation for multilingual tasks . authors often examine models with one data set that is representative of all possible data . |
| Approach: | They compare three broad classes of models with different parameterizations using morphological segmentation as the test case. |
| Outcome: | The results show that the extent of model generalization depends on the characteristics of the data set, and does not necessarily rely heavily on the data sets size. |
On Synthetic Data for Back Translation (2022.naacl-main)
Copied to clipboard
| Challenge: | Existing studies on back translation (BT) focus on beam search or random sampling . a new method to generate synthetic data with a backward model is proposed to improve BT performance. |
| Approach: | They propose a method to generate synthetic data to trade off quality and importance factors . back translation (BT) is one of the most significant technologies in NMT research fields . |
| Outcome: | The proposed method outperforms the baseline methods on WMT14 DE-EN, EN-DE, and RU-EN benchmark tasks. |
Efficient Multi-Task Auxiliary Learning: Selecting Auxiliary Data by Feature Similarity (2021.emnlp-main)
Copied to clipboard
| Challenge: | Multi-task auxiliary learning uses a set of relevant auxiliary tasks to improve performance of a primary task. |
| Approach: | They propose a time-efficient sampling method to select the most beneficial sub-datasets from the auxiliary tasks to achieve efficient multi-task auxiliary learning. |
| Outcome: | The proposed method significantly outperforms random sampling and ST-DNN on three benchmark datasets. |
Building a Corpus from Handwritten Picture Postcards: Transcription, Annotation and Part-of-Speech Tagging (L18-1)
Copied to clipboard
| Challenge: | In this paper, we describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards. |
| Approach: | They describe the processes and challenges of digitalisation, manual transcription, and manual annotation of over 11,000 postcards written in German and Swiss German. |
| Outcome: | The proposed system outperforms state-of-the-art taggers in the evaluation of the 'picture postcard corpus' containing over 11,000 handwritten postcards . |
Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation (D18-1)
Copied to clipboard
| Challenge: | Neural machine translation (NMT) uses a sequence-to-sequence model to generate synthetic data. |
| Approach: | They propose a method that adds synthetic data to sentences with high prediction loss during training and a variety of sampling strategies targeting difficult-to-predict words. |
| Outcome: | The proposed method improves translation quality by up to 1.7 and 1.2 Bleu points over back-translation using random sampling for German-English and English-German, respectively. |
Detection of Abusive Language: the Problem of Biased Datasets (N19-1)
Copied to clipboard
| Challenge: | Recent studies have reported high classification performance on datasets with difficult cases of abusive language. |
| Approach: | They examine the impact of data bias on abusive language detection by focusing on specific microposts rather than random sampling. |
| Outcome: | The proposed method is more accurate and more accurate than random sampling. |
Active Learning for Corpus Refinement: Cost-Effective Preprocessing to Improve Validity of Applied Quantitative Text Analysis (2026.eacl-srw)
Copied to clipboard
| Challenge: | Quantitative text analysis relies on high-quality corpora, but keyword-based collection often retrieves irrelevant material, undermining validity. |
| Approach: | They propose to use a transformer-based classifier to iteratively refine corpora by excluding irrelevant documents. |
| Outcome: | The proposed method outperforms random sampling and weakly supervised sampling and outperformed random sampling. |
Achieving Model Robustness through Discrete Adversarial Training (2021.emnlp-main)
Copied to clipboard
| Challenge: | Discrete adversarial attacks are symbolic perturbations to a language input that preserve the output label but lead to predicting error. |
| Approach: | They propose a discrete adversarial attack based on best-first search and random sampling attacks that are not based upon expensive search procedures. |
| Outcome: | The proposed attack outperforms offline augmentation and speedups on three datasets. |
Automatic Detection of Generated Text is Easiest when Humans are Fooled (2020.acl-main)
Copied to clipboard
| Challenge: | Recent advances in neural language modelling make it possible to rapidly generate vast amounts of human-sounding text. |
| Approach: | They compare decoding methods with popular sampling-based decoding strategies . they show that multi-sentence excerpts can fool expert human raters over 30% of the time . |
| Outcome: | The proposed methods improve with longer excerpt length, but multi-sentence excerpts fool human raters over 30% of the time. |
Unlocking the Planning Capabilities of Large Language Models with Maximum Diversity Fine-tuning (2025.findings-naacl)
Copied to clipboard
| Challenge: | Existing studies have shown that LLMs struggle to generate valid plans in the automated planning domain due to weak System 2 competencies. |
| Approach: | They propose a method which uses a random sampling method to select diverse and representative data to enhance sample efficiency and the model’s generalization capability. |
| Outcome: | The proposed method outperforms baseline methods across scales and multiple benchmark domains. |
Grouping Words with Semantic Diversity (2021.naacl-main)
Copied to clipboard
| Challenge: | a new approach to grouping input words based on their semantic diversity is proposed . high-dimensional inputs and learning complexity hinders deep learning generalization, authors say . |
| Approach: | They propose a way to group input words based on their semantic diversity to simplify input language representation with low ambiguity. |
| Outcome: | The proposed methods generalize NLP models and demonstrate improvements on medium-scale machine translation tasks. |
On the Limitations of Simulating Active Learning (2023.findings-acl)
Copied to clipboard
| Challenge: | Active learning (AL) is a human-and-model-in-the-loop paradigm that iteratively selects informative unlabeled data for human annotation. |
| Approach: | They propose to simulate active learning by using an already labeled dataset as the pool of unlabeled data. |
| Outcome: | The proposed model-in-the-loop paradigm can be used to perform experiments with human annotations on-the fly. |
Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning (2026.eacl-long)
Copied to clipboard
| Challenge: | Curriculum learning has improved efficiency across machine learning domains, but remains underexplored for language model pretraining. |
| Approach: | They present a systematic investigation of curriculum learning in LLM pretraining . they use vanilla curriculum learning, pacing-based sampling, and interleaved curricula . |
| Outcome: | The proposed framework accelerates convergence in early and mid-training phases, reducing training steps by 18-45% to reach baseline performance. |
Active Learning Principles for In-Context Learning with Large Language Models (2023.findings-emnlp)
Copied to clipboard
| Challenge: | In-context learning has significantly enhanced predictive performance in few-shot learning settings. |
| Approach: | They propose to use pool-based Active Learning to identify the most informative demonstrations for few-shot learning over a single iteration to identify best demonstrations. |
| Outcome: | The proposed model outperforms all other methods, including random sampling, in the analysis of 24 classification and multi-choice tasks. |
An Experimental Design Framework for Label-Efficient Supervised Finetuning of Large Language Models (2024.findings-acl)
Copied to clipboard
Gantavya Bhatt, Yifang Chen, Arnav Das, Jifan Zhang, Sang Truong, Stephen Mussmann, Yinglun Zhu, Jeff Bilmes, Simon Du, Kevin Jamieson, Jordan Ash, Robert Nowak
| Challenge: | Supervised finetuning (SFT) on instruction datasets has shown immense potential in improving the zero-shot generalization capabilities observed in large language models (LLMs). |
| Approach: | They propose to use experimental design to minimize the computational cost of active learning by identifying useful subsets of samples to annotate from an unlabeled pool. |
| Outcome: | The proposed methods save 50% of the annotation cost compared to random sampling on generative tasks. |
Deep Reinforcement Learning with Hierarchical Action Exploration for Dialogue Generation (2024.lrec-main)
Copied to clipboard
| Challenge: | Existing approaches to improve dialogues with random sampling are inefficient due to the large number of eligible responses with high action values. |
| Approach: | They propose a dual-granularity Q-function that extracts actions based on a grained hierarchy . they use offline RL and learn from multiple reward functions designed to capture emotional nuances in human interactions. |
| Outcome: | The proposed approach outperforms baselines across automatic metrics and human evaluations. |
What data should I include in my POS tagging training set? (2025.findings-emnlp)
Copied to clipboard
| Challenge: | POS tagging is a crucial task for descriptive linguistics and language documentation . POS tags are not available in all languages, but are used for training sets for understudied languages . |
| Approach: | They compare POS tagging with in-context learning, active learning, and random sampling . they find that POS can deliver reasonable results for communities with limited resources . |
| Outcome: | The proposed training set for Indigenous and endangered languages performs better than random sampling. |
Mitigating Demonstration Bias through Global Coevolutionary Reasoning (2025.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for chain-of-thought prompting rely on manual demonstrations . experimental results show that GCR outperforms baseline methods without performance degradation . |
| Approach: | They propose a method that uses random samples to generate demonstrations in zero-shot settings. |
| Outcome: | The proposed method outperforms baseline methods on ten datasets without demonstration bias. |
Generating Diverse and High-Quality Texts by Minimum Bayes Risk Decoding (2024.findings-acl)
Copied to clipboard
| Challenge: | Existing decoding algorithms to generate diverse outputs are based on beam search or random sampling, thus their output quality is capped by these underlying decoding methods. |
| Approach: | They propose to add a diversity penalty to MBR decoding and a clustering problem to create diversity-promoting decoding algorithms by enforcing diversity objectives. |
| Outcome: | The proposed method achieves a better trade-off than the diverse beam search and sampling algorithms overall. |
MAGRET: Machine-generated Text Detection with Rewritten Texts (2025.coling-main)
Copied to clipboard
| Challenge: | Existing studies focus on detecting machine-generated text in open-source models, but their performance on closed-source large models is limited. |
| Approach: | They propose a method to detect rewritten text from large language models using a BERT encoder and propose to refine it to achieve semantic alignment. |
| Outcome: | The proposed method outperforms baseline methods on three text-generated datasets. |
MeaeQ: Mount Model Extraction Attacks with Efficient Queries (2023.emnlp-main)
Copied to clipboard
| Challenge: | Recent studies focus on limited-query budget settings and adopt random sampling or active learning-based sampling strategies on publicly available, unannotated data sources. |
| Approach: | They propose a model extraction attack with efficient Queries that uses a zero-shot sequence inference classifier to filter task-relevant data from a public text corpus instead of a problem domain-specific dataset. |
| Outcome: | The proposed method achieves higher similarity to the victim model than baselines while requiring fewer queries. |
Balanced Data Sampling for Language Model Training with Clustering (2024.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models (LLMs) are a fundamental part of the training process. |
| Approach: | They propose to use clustering to balance the text distribution of training data for better model training. |
| Outcome: | Extensive experiments validate the effectiveness of ClusterClip Sampling under various training datasets and large language models. |
DARS: Dynamic Action Re-Sampling to Enhance Coding Agent Performance by Adaptive Tree Traversal (2025.acl-long)
Copied to clipboard
| Challenge: | Existing approaches to developing LLM-powered coding agents struggle with sub-optimal decision-making. |
| Approach: | They propose a novel inference time compute scaling approach that recovers from sub-optimal decisions by branching out a trajectory at certain key decision points by taking an alternative action given the history of the trajectory and execution feedback of the previous attempt. |
| Outcome: | The proposed approach achieves a pass@1 rate of 47% on the SWE-Bench Lite benchmark, outperforming state-of-the-art (SOTA) open-source frameworks. |
Rebalancing Label Distribution While Eliminating Inherent Waiting Time in Multi Label Active Learning Applied to Transformers (2024.lrec-main)
Copied to clipboard
| Challenge: | Data annotation is a resourceintensive endeavor, necessitating human involvement and expertise. |
| Approach: | They propose to annotate instances to rebalance label distribution by judiciously selecting and limiting the data to be annotated. |
| Outcome: | The proposed method mitigates biases, improves model performance and reduces strategy-dependent disparities. |
Preference-Guided Reflective Sampling for Aligning Language Models (2024.emnlp-main)
Copied to clipboard
| Challenge: | Repeated random sampling is a widely used method that independently queries the model multiple times to generate outputs. |
| Approach: | They propose a more efficient method for iterative data generation and model re-training that leverages tree-based tree-derived generation framework to enable more efficient sampling. |
| Outcome: | The proposed method significantly outperforms repeated random sampling in best-of-N sampling on AlpacaEval and Arena-Hard. |
On the Fragility of Active Learners for Text Classification (2024.emnlp-main)
Copied to clipboard
| Challenge: | Active learning (AL) techniques optimally utilize a labeling budget by iteratively selecting instances that are most valuable for learning. |
| Approach: | They propose to use active learning techniques to iteratively select instances that are most valuable for learning. |
| Outcome: | The proposed framework is used to benchmark active learning techniques for text classification using pre-trained representations. |
S3Prompt: Instructing the Model with Self-calibration, Self-recall and Self-aggregation to Improve In-context Learning (2024.lrec-main)
Copied to clipboard
| Challenge: | Large language models have limitations in practical applications, such as unsupervised generation and recall of in-context examples. |
| Approach: | They propose a self-calibration, self-recall and self-aggregation prompt pipeline to solve these problems. |
| Outcome: | The proposed pipeline improves the performance of large language models without annotating datasets and model parameter updates. |
A Diversity Diet for a Healthier Model: A Case Study of French ModernBERT (2026.findings-acl)
Copied to clipboard
| Challenge: | Diversity has been gaining interest in the NLP community in recent years. |
| Approach: | They propose to use diversity-driven sampling to pre-train models on French with a fixed compute budget. |
| Outcome: | The diversity-driven sampling reduces the pre-training dataset by 94% and the pretraining time by 73% while maintaining comparable performance. |
Detecting Corpus-Level Knowledge Inconsistencies in Wikipedia with Large Language Models (2025.emnlp-main)
Copied to clipboard
Sina Semnani, Jirayu Burapacheep, Arpandeep Khatua, Thanawan Atchariyachanvanit, Zheng Wang, Monica Lam
| Challenge: | a new study examines the accuracy of Wikipedia's factual inconsistencies . a corpus-level inconsistent detection system can help editors identify inconsistances . |
| Approach: | They propose a corpus-level inconsistency detection system that combines LLM reasoning with retrieval to detect and contextualize potential contradictions for human review. |
| Outcome: | The proposed system can detect inconsistencies in Wikipedia and human review. |